A Multi-Objective Genetic Algorithm Approach to Feature Selection in Neural and Fuzzy Modeling

A large number of techniques, such a neural networks and neurofuzzy systems, are used to produce empirical models based in part or in whole on observed data. A key stage in the modelling process is the selection of features. Irrelevant or noisy features increase the complexity of the modelling problem, may introduce additional costs in gathering unneeded data, and frequently degrade modelling performance. Often it is acceptable to trade off some decrease in performance against a reduction in complexity (number of input features), although we rarely know a priori what an acceptable trade-off is. In this paper, feature selection is posed as a multiobjective optimisation problem, as in the simplest case it involves feature subset size minimisation and performance maximisation. We propose multiobjective genetic algorithms as an effective means of evolving a population of alternative feature subsets with various modelling accuracy/complexity trade-offs, based on the concept of dominance. We discuss methods to reduce the computational costs of the technique, including the use of special forms of neural network and neurofuzzy models. The major contributions of this paper are: the formulation of feature selection as a multiobjective optimisation problem; the use of multiobjective evolutionary algorithms, based on the concept of dominance, for multiobjective feature subset selection; and the application of the multiobjective genetic algorithm feature selection on a number of neural and fuzzy models together with fast subset evaluation techniques. By considering both neural networks and neurofuzzy models, we show that our approach can be generically applied to different modelling techniques. The proposed method is applied on two small and high dimensional regression problems.